Boosted Dynamic Neural Networks

Early-exiting dynamic neural networks (EDNN), as one type of networks, has been widely studied recently. A typical EDNN multiple prediction heads at different layers the network backbone. During inference, model will exit either last head or an intermediate where confidence is higher than a predefined threshold. To optimize model, these together with backbone are trained on every batch training data. This brings train-test mismatch problem that all optimized types data in phase while deeper only see difficult inputs testing phase. Treating and differently two phases cause between distributions. mitigate this problem, we formulate additive inspired by gradient boosting, propose techniques to effectively. We name our method BoostNet. Our experiments show it achieves state-of-the-art performance CIFAR100 ImageNet datasets both anytime budgeted-batch modes. code released https://github.com/SHI-Labs/Boosted-Dynamic-Networks.